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ABSTRACT 

For more than twenty years accrediting agencies have required assessment as part of their 
initial accreditation or reaffirmation processes. During that period of time thousands of 
institutions have successfully prepared plans to achieve or maintain their accreditation. Why 
then does a culture of assessment not exist? And why is assessment still an issue of contention in 
most colleges and universities? Using existing research one School of Business has been 
following a change management approach to foster a culture of assessment. The process 
followed and the journey are described. Recommendations are made so that others can avoid 
some of the pitfalls encountered as the assessment process was improved. 
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INTRODUCTION 

For more than twenty years regional and professional accrediting agencies have required 
assessment as part of their initial accreditation or reaffirmation processes. During that period of 
time thousands of institutions have prepared self-studies, AQIP Plans, Quality Enhancement 
Plans, Initial Accreditation Plans, and a variety of other plans to achieve or maintain their 
accreditation. The vast majority of these institutions have been successful in achieving their 
accreditation goals. Why then does a culture of assessment not exist? And why is assessment 
still an issue of contention in most colleges and universities? 

There are two answers to these questions. First, the definition of the concept has changed 
over time. Early in this period most of the activities related to assessment were conducted by 
department chairs, associate deans, deans, and other administrators. Much of the work involved 
in this process was collection of secondary data from students, alumni and employers of 
graduates. These processes survived as acceptable for a few years as institutions adapted to these 
new requirements. Second, the next phase in the accreditation process related to assessment 
involved development of plans to measure student learning at a program-level. Although the 
development of these plans involved faculty participation, this participation was normally mostly 
a review of plans developed by administrators such as a director of assessment or small sub 
committees. Again, for a period of years institutions which had successfully completed this 
process achieved accreditation or reaffirmation. 

The next phase in the accreditation process related to assessment involved actually 
implementing the plans which were developed. Many implementation teams were composed of 
a small number of faculty members who either went along reluctantly or believed in the process 
and accepted the responsibility willingly. Admittedly, many of the processes and measurements 
were rudimentary and did not involve significant time or effort to complete. The cycle of using 
the data to make changes and close the assessment loop was rarely completed in this phase. 

Over the last ten years both assessment activities and assessment terminology have 
changed. Now, rather than requiring “assessment” most agencies use terminology such as 
“student learning outcomes,” “assurance of learning,” “institutional effectiveness,” “data driven 
continuous improvement” or other similar terminology. In addition to the change in 
terminology, there has been an associated change in the expectations of the accrediting agencies. 
Now, when teams visit campuses they expect to see evidence that the appropriate measurements 
are occurring at the program level (including general education or general studies programs). In 
addition, visiting teams are looking for evidence that the information gained from measuring the 
student learning outcomes has been used to close the loop and improve the curriculum within 
each program to ensure program learning goals are being achieved at an appropriate level. The 
continuous process of assessment allows measurement of the changes in learning outcomes that 
have come about because of the changes to the curriculum. This is the process of continuous 
improvement. 

As a result of increasing requirements and pressures from the federal government, many 
governing boards and state legislatures have entered the fray and are requiring reports on the 
results of these activities, especially as they relate to ensuring sufficient learning outcomes for 
programs. As those in the academy work to meet the requirements of this ever-expanding list of 
significant constituents, several additional challenges impact our ability to achieve these goals. 
Some of these challenges include: the graying of the faculty (nearly 50% of college and 
university faculty will be approaching retirement age in the next ten years); decreased funding 
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(especially at public institutions); lack of or very small faculty compensation enhancements 
(raises); increased research requirements for tenure and promotion; and finally the dramatic 
expansion of alternative delivery systems (especially on-line and other technologically delivered 
programs). 

LITERATURE REVIEW 

In the current higher education environment, the most common comments heard from 
faculty members regarding the requirement of measuring student learning outcomes and revising 
curricula relate to workload. It is very common to hear a statement such as, “this is just another 
responsibility the administration is asking faculty members to accept with no change in workload 
and no increase in compensation.” Although most institutions have made some significant 
inroads into implementing successful programs for assessing student learning outcomes at the 
program level, there are few institutions which could claim a serious campus-wide commitment 
to the entire process. 

Outcomes-based assessment processes are detailed in a number of manuals and 
handbooks, such as Palomba & Banta, 1999; Banta, 2002; Bresciani, Zelna, & Anderson, 2004; 
and Suskie, 2009, to name a few. The publications not only define assessment, but also 
distinguish assessment from grades, program reviews, institutional effectiveness, and key 
performance indicators (Huba & Freed, 2000; Suskie, 2009). The steps involved in implementing 
outcomes-based assessment begin with defining learning goals and conclude with making 
changes to improve student learning (Huba & Freed, 2000; Martell & Calderon, 2005; Gardiner, 
Corbitt, & Adams, 2010). However, as Marques and Garrett (2012) state, “Simply introducing 
resistant faculty to a model.. ..does not lead to sustainable change.” 

Part of this change involves a switch to assessment of an entire program instead of 
individual majors or courses (Marques & Garrett, 2012). For some faculty, this change in focus 
is the basis for the resistance. Several other factors challenge the implementation of an 
outcomes-based assessment program. Challenges for faculty and administrators include time 
constraints, resource constraints, and lack of understanding of assessment, its defined purpose 
and implementation strategies (Bresciani, 2011). If faculty do not know how to do assessment, 
they are unlikely to take the time to learn on their own. Arguments about reasons not to do 
assessment or to not follow someone else’s prescribed procedures are prevalent (Martell, 2005). 

The shift towards assessment has some of the trials and tribulations of a paradigm shift: 
faculty do not trust the need to change or the people delivering the message of change; faculty 
persist in teaching the way they have always taught, believing that it is sufficient for making a 
difference; faculty fear losing their identity if they have to change the way they do things or the 
tools they use (Huba & Freed, 2000). Frequently a change agent or champion must serve as the 
catalyst for change during the assessment process (Martell, 2005). However, being the champion 
or change agent for a paradigm shift may create distance from colleagues, or may result in 
working harder than colleagues in the struggle for asking and answering the questions required to 
get started (Huba & Freed, 2000). 

In spite of these difficulties, journals on assessment are now commonplace in which 
schools report on successful efforts in implementing an assessment plan. Examples include 
Journal of Education for Business; Practical Assessment, Research and Evalution; Research and 
Practice in Assessment; Journal of Case Studies in Accreditation and Assessment; and 
Assessment & Evaluation in Higher Education. The degree of success in implementation 
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depends on leadership that supports assessment and outlines expectations, funding for faculty 
development and training, and structure that often includes release time or other incentives and 
rewards for faculty to participate (Huba & Freed, 2000; Martell, 2005; Bresciani M. J., 2006a; 
Kramer, Hanson, & Olsen, 2010; Bresciani M. J., 2011). Responsibilities of the faculty and staff 
are explicit in the most successful cases (Popper, 2005; Bresciani M. J., 2006a; Stivers & 

Phillips, 2009). Successful implementation is more likely to be achieived with a faculty-directed 
process. A greater voice in the assessment process results in the faculty taking more of a personal 
interest in the performance of the students and greater receptivity towards making changes to 
improvements in student learning (Kidwell & Peek, 2003; Anderson-Fletcher, 2005; Stivers & 
Phillips, 2009). Furthermore, a focus on improving student learning as opposed to meeting 
accreditation requirements is deemed important for the success of implementation (Marques & 
Garrett, 2012). 

The assessment process will be more widely adopted when the organization supports 
assessment and creates a culture of assessement. A culture of assessment means that the concept 
of assessment is widely and clearly understood (Huba & Freed, 2000) and faculty and staff share 
a belief in a systematic process for continuous improvement (Gray, 2010). It depends on open 
communication, extensive faculty development, faculty ownership of the process, permission to 
fail but not to stall, and experimentation (Eder, 2005). Transforming an assessment plan from a 
disruption to a normal activity requires a meaningful, manageable, and sustainable process 
(Gray, 2010). Commitment by leadership, adequate resources, shared responsibility, measuring 
what is valued and using results to drive decision-making creates a meaningful assessment 
experience for constituents (Kramer, Hanson, & Olsen, 2010). Manageability is enhanced 
through strong, consistent, and sensitive communication about the importance of and the plan for 
assessment implementation (Gray, 2010). Sustainability needs effective leadership for providing 
the rationale for assessment, allaying fears, engaging the campus community, and providing 
intrinsic and extrinsic incentives for faculty and staff (Gray, 2010). 

Increased accountability of student learning at the board-level, and state and federal- 
levels will continue into the foreseeable future. As a result there continues to be a significant 
need to improve the campus-wide environment for effective implementation of data driven 
decision processes for each program. When faculty members hear they are expected to 
participate in assessment activities, they have the same reaction that most people do when they 
are approached about a change - something along the lines of “I’m supposed to do what? And 
WHY on earth would anyone do that?” These reactions are normal and to be expected when 
change is the issue. Using existing research one School of Business has been following a change 
management approach to foster a culture of assessment in the school. The journey they took is 
outlined below. Recommendations to avoid some of the pitfalls encountered are included. 

MANAGING THE CHANGE 

Irrespective of what is thought about the progress made in improving assessment at 
academic institutions, the processes have not been institutionalized. As a result many faculty 
members still do not see the need nor the urgency in implementing student learning outcomes 
assessment and more importantly, faculty members often continue to resist the process of closing 
the loop with feedback to curricular offerings and coverage. Informed by research, it is 
commonly understood that people do not like change. Some faculty actively oppose change 
while others react much more passively and simply continue their status quo. In order to create 
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a campus-wide culture of student learning outcomes assessment, both of these groups of faculty 
members must be converted to participants. Despite the challenges outlined earlier, this change 
can happen. It must be intentional and continuous. As a result Kotter’s (1996) approach to 
change management was adopted. 

Kotter (1996) suggests the following eight steps in the change management process. He 
suggests that it is a process, and that all 8 steps must be followed. These steps are used to outline 
the process followed to create a culture of assessment. 

1. Establishing a Sense of Urgency 

2. Forming a Powerful Guiding Coalition 

3. Developing a Vision and Strategy 

4. Communicating the Vision 

5. Empowering Others to Act on the Vision 

6. Planning for and Creating Short-Term Wins 

7. Consolidating Improvements and Producing Still More Change 

8. Institutionalizing New Approaches 

Establishing a Sense of Urgency 

In the past, accreditation processes have been relied upon heavily to help establish a 
sense of urgency among faculty members. This has worked well with two groups of faculty 
members on most campuses. Those serving on institutional committees with the responsibility 
for leading reaffirmation efforts for regional accreditation have accepted the responsibility and 
have gained an understanding of the requirements. Faculty members in programs which have 
achieved or are seeking to achieve professional accreditation have also typically accepted the 
responsibility. Unfortunately, this still represents a minority of faculty members on most 
campuses. 

The ability to create a sense of urgency for other faculty members has been enhanced by 
the broadening of those constituents to whom universities report. Governing boards of public 
institutions have begun to attach funding or funding increases or decreases to a successful 
demonstration of sufficient student learning outcomes. In these difficult economic times, any 
issue tied to the potential reduction of funding should be a motivating factor for faculty 
members. The requirements for enhancing student learning are not going away any time soon. 

As institutions endure continued decreases in funding, program eliminations, faculty lay-offs, 
and furloughs, the motivation to ensure the quality of programs through well documented student 
learning outcomes increases for the faculty members in those programs. 

Forming a Powerful Guiding Coalition 

In the change management process, Kotter (1996; p57) identifies four important 
characteristics for members of the guiding coalition (initially in this case, the School of Business 
Assessment Committee). These characteristics are: 

1. Position power - with sufficient numbers to overcome opposition; 

2. Expertise - with sufficient breadth to represent many points of view; 

3. Credibility - individuals sufficiently well respected by their colleagues to be taken 

seriously; 

4. Leadership - proven leaders of both action and opinion. 
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The charge of this guiding coalition is important. Their charge should make it clear that the 
responsibility of this group is to lead the charge to improve student learning outcomes 
assessment for the program. It should be clear that this group is providing leadership, but that 
the responsibility for achieving the results is shared by all the faculty members within the 
program. 

The School of Business Assessment Committee was led by the Associate Dean who was 
well respected and had been in this role for more than fifteen years. With the two department 
chairs, the director of the Business Advising Center, six faculty members and two students, all 
academic disciplines in the school and a third of the faculty were represented. The 
characteristics listed above were all met by the committee; there was a powerful guiding 
coalition in place. 

Developing a Vision and Strategy 

In the third step of the process, Kotter (1996) identifies the importance of developing a 
vision and strategy. This becomes one of the most critical functions of the guiding coalition. 
Kotter indicates the importance of clarifying the direction of change (1996; p69) as this is the 
variable upon which most people disagree. He indicates that an effective vision and strategy will 
communicate the following: “This is how our world is changing, and here are compelling 
reasons why we should set goals and pursue these new ... quality programs.” (1996; p69) 

In the context of student learning outcomes assessment, what does it mean to create a 
vision and strategy? Since one important aspect of developing a vision relates to identity, the 
group immediately changed the committee’s name. The long-standing and poorly understood 
Assessment Committee was replaced by the Student Learning Committee to better describe the 
function they were leading. The next step in the visioning process was to identify the 
characteristics of the ideal business graduate. In other words a great deal of work was done by 
the guiding coalition to identify the most important skills, knowledge, and capabilities each 
graduate of the business program should possess. In gathering this information, the guiding 
coalition sought input from the School’s stakeholders including students and members of the 
Executive Advisory Council. The learning goals developed through the process were shared 
with the Advisory Council, students, and faculty members. Input was solicited at each level and 
the resulting goals were approved by the faculty. Specific objectives and scoring rubrics were 
developed by the faculty during an Assessment Forum which was held each semester. Once the 
School’s faculty voted to accept these important outcomes, the vision and strategy were set. 

The most important component of the strategy development was the Guiding Coalition’s 
commitment to provide good evidential support identifying the areas of strength and any areas 
needing improvement related to graduates. The Coalition also determined that gathering this 
evidence must be accomplished with as minimal an impact on faculty workload as possible. The 
Student Learning Committee embarked on a strategy to embed as much of the assessment into 
coursework as was reasonably possible. This was done in order to minimize the resistance by 
faculty members due to workload increases. For one learning goal, the Committee determined 
that the best alternative to assess the students’ overall business knowledge was the use of a 
standardized test of business. This direction provided two significant challenges. First, where in 
the curriculum would this exam fit and be a reasonable measure of student learning? The second 
challenge was how to get the students to take the test seriously? The committee ultimately 
formed a vision that the test would be administered as part of the capstone Business Policy class 


Improving assessment: creating a culture, page 6 


Journal of Case Studies in Accreditation and Assessment Volume 4 - December, 2014 


at both the undergraduate and graduate levels. In order to implement this recommendation, 
changes in graduation requirements and in two courses would be required at the undergraduate 
level. The buy-in of the faculty members teaching the courses at both levels was also required. 
For other learning goals, embedded assessment was the agreed upon approach including using 
existing assignments that corresponded with a specific goal and incorporating the Business 
Critical Thinking Skills Test (BCTST) from Insight Assessment in class to assess critical 
thinking. The school continued to use the existing indirect measures including alumni and 
graduating senior surveys. 

Communicating the Vision 

Once the vision and strategy were determined, the Committee began a process of 
communicating the vision and strategy through a variety of mechanisms. First, due to the 
requirement of curricular change at the undergraduate level, the business faculty as a whole was 
asked to consider a change in the undergraduate graduation requirements for all undergraduate 
business majors. This requested change would create a zero-credit hour class on a pass/fail 
basis. This class would be an automatic co-requisite with the undergraduate Business Policy 
class. The additional change in the graduation requirements was to set the requirement that 
students must receive a pre-determined minimum acceptable score on the test in order to receive 
an acceptable grade in the zero-credit hour class in order to graduate. The second set of changes 
which related specifically to the Business Policy class was an agreement regarding the impact of 
the exam results on the final grade in the Business Policy class. Discussion of this resulted in a 
scale of scores ranging from minus five percent to plus five percent of the final grade in the 
Business Policy class based upon the individual results of the standardized exam. Other 
communication revolved around including all faculty members in the assessment of embedded 
activities. 

Clearly this set of curricular and graduation changes were developed to resolve the issue 
of students taking the exam seriously. It is only fair to say that there is still discussion among 
faculty members regarding whether or not it is appropriate to keep a student from graduating 
based upon the score of a standardized exam. The Student Learning Committee developed and 
the faculty approved an alternative assessment process, including review materials, for the very 
small number of students who could not achieve the minimum score on the standardized test. In 
addition to the graduation and grade impacts of the test, the School agreed to provide a financial 
reward to the top five scores on the test each semester to further incentivize the students. 

Interestingly, the graduate faculty decided that there was no need to make the exam 
results either a graduation requirement or a component of the grade in the Graduate Business 
Policy and Strategic Management class. The faculty members teaching this course agreed to use 
one three-hour class period to administer the exam and again a financial reward and recognition 
were offered to the highest five scores. The results of the exam were tracked in a spreadsheet. 
Training was provided to ensure faculty members were appropriately interpreting results and 
determining the statistical validity of the results. 

The remaining goals were assessed using learning activities embedded in regular business 
core courses for both the undergraduate and the graduate degree programs. These learning 
activities included writing assignments, case analyses, problem solving techniques and skill 
development. After the activities were collected, committee members enlisted the help of other 
faculty members outside the committee to conduct the actual assessment. Each activity was 
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assessed by three individuals in the School of Business. The results from the embedded 
assessments were tracked in a spreadsheet and averaged. 

In order to ensure effective assessment of these activities, faculty members participating 
were trained. First and foremost, ensuring a common understanding of the rubrics and how to 
apply them consistently was essential to creating internal cross-evaluator validity. 

Empowering Others to Act on the Vision 

Continuing the process of communicating the vision and moving into the stage of 
“Empowering Others to Act on the Vision” the Committee determined it was time to get 
everybody involved at a fundamental level in the student learning outcomes process. Faculty 
leaders in the School were asked to chair a task force. Every faculty member was assigned to a 
task force with a good distribution of disciplines and experience on each task force. Each task 
force was assigned one undergraduate and one graduate learning goal and was charged with 
reviewing, analyzing, and discussing the assessment data that had been collected for the goals. 
The Task Force was to develop several recommendations for closing the loop and improving 
student learning where weaknesses were identified. 

Planning for and Creating Short-Term Win s 

Of course the acceptance of the work to this point by the faculty as a whole required a 
celebration which the School undertook in a modest way with a pot luck dinner. In fact, in 
hindsight the celebration should probably have been much larger as the plan to this point was a 
significant improvement over previous assessment plans in the School. This was only an interim 
celebration as the implementation process was still to come. The next phase of the work was the 
process of mapping the objectives to the core (required) courses for business majors. A matrix 
was developed that included the list of learning goals and objectives down the left hand side and 
the list of core courses across the top. Faculty who taught the core classes were asked to assess 
the level of inclusion of the topic on a five point scale as follows: 

5 = Stated course objective 
4 = Includes a significant learning experience 
3 = Includes a moderate learning experience 
2 = A minor component of the course 
1 = Incidental reference or experience in the course 
*= Not included in the course 

The task forces took the summarized results of this data collection and assessed whether 
there was appropriate content included for each goal. In cases where coverage was insufficient, 
discipline faculty were asked to make adjustments to their classes to improve content coverage 
relative to the learning goals and objectives where necessary. Duplication of coverage was 
considered a strength of the curriculum, and no action was taken to minimize duplicated 
coverage of objectives. The results of this process provided another victory and another 
opportunity to celebrate. The first iteration of “closing the loop,” changing the curriculum as a 
result of assessment activities was complete and another small celebration took place. 
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Consolidating Improvements and Producing Still More Change 

As the official AACSB visit for reaffirmation of accreditation loomed on the horizon, an 
AACSB consultant came for a visit. He was very complimentary of the process, but made it 
very clear the School was at risk due to the minimal amount of data that had been collected. His 
visit helped to reinforce the sense of urgency (step one above!) surrounding assessment. He was 
particularly encouraging of the senior faculty to lead the assessment effort. 

After the consultant’s visit, to maximize the number of data points, a plan was 
established to collect and evaluate data each semester (in most cases it is customary practice to 
collect data once per year). Each activity for which data was collected was evaluated using the 
established rubrics by three individuals. This process was coordinated by the chair of the 
Student Learning Committee in the beginning. As the task of lining up three individuals for each 
embedded activity became onerous for the one individual to oversee, the Committee suggested 
the responsibilities be distributed and that someone from the Committee take the data from the 
embedded activity and be responsible for enlisting two additional faculty volunteers to assess the 
data creating the groups of three mentioned above. This idea was discussed and explained in a 
School of Business meeting so that everyone was aware of their impending involvement. 

The Associate Dean’s office maintained records of who had the data and stored the actual 
data (student work and faculty assessment) when each review was completed. The assessment 
data was entered into a spreadsheet format. As this process accelerated, the need for an 
electronic data management system for recording, storing, manipulating, and reporting became 
very evident. As a result, the School is cooperating and helping to lead the implementation of an 
effective data management system for assessment on campus. 

- 

Institutionalizing New Approaches 

The task forces met again to discuss the results of the embedded assessments. The results 
of the task force discussions were recorded and compiled into recommendations by the Student 
Learning Committee. The recommendations for closing the loop to improve student learning led 
to curriculum and policy changes that were proposed to the faculty for a vote at a School of 
Business meeting. A few examples of the changes that were approved include changes to 
course prerequisites and changes in course syllabi to include information on the standardized 
exam required for graduation. This closing the loop process included everyone in the School of 
Business. The culture of assessment was at its strongest immediately before the successful 
reaccreditation visit. 

When the AACSB reaccreditation visit took place, the first thing the team asked about 
was assessment. They were presented with student work and assessment data along with the 
spreadsheet. The team selected several pieces of student work to review in light of the results of 
assessment activities. They examined the student work and the assessments to ensure the 
documents were genuine. They asked for one other sample and were pleased with what they 
saw. The School was complemented on a job well done. Quoting from the letter of 
reaffirmation, “In an era when assessment and outcomes are being scrutinized by the public and 
government entities, the school has embraced these changes .... The assessment framework is 
strong in that it utilizes direct and indirect measures, has multiple modes of evaluation, and 
gathers input from key stakeholders (students, faculty and business community).” 
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SUMMARY AND RECOMMENDATIONS 

Realizing the importance of assurance of learning, Kotter’s (1996) method of change 
management was utilized to create a culture of assessment. Each of the phases in Kotter’s 
method was followed with positive results. The sense of urgency was real and enhanced by the 
AACSB consultant. While it would be nice to report that ensuring student learning was the key 
factor in establishing the sense of urgency, the truth is that it was the AACSB reaccreditation that 
was the main factor. Kotter (1996) includes as an example a CEO who took a knife to a tire to 
establish a sense of urgency -the interpretation is to use what is at hand to create the sense of 
urgency. A guiding coalition led the process and involved everyone in the School of Business in 
developing the vision and strategy. The vision was communicated often and in various ways. 
Others were empowered (enlisted) to act on the vision and wins were celebrated. Suggestions 
were consolidated and discussed and improvements were voted on by the faculty. The School of 
Business became involved with the implementation of an effective data management system for 
assessment on campus. Assessment is no longer a four letter word in the halls - assurance of 
learning and student learning committee are phrases heard instead of assessment and without 
negative connotation. 

Each institution has its own culture and its own faculty who are resistant to change and 
assessment for any number of reasoni. as discussed in the literature review. While there is not a 
one-size-fits-all approach to creating a culture of assessment, following Kotter’s (1996) process 
is close. Kotter (1996) encourages those who follow his process to be sure to not skip any of the 
steps. Establishing the sense of urgency is a first step, but it is not enough. Those who think 
assessment is a short-term fad should be convinced by now that this is simply not the case. 
Communication is key. It is-important to faculty to hear the message multiple times in multiple 
ways and from multiple people in authority. Celebrating can help keep the message in front of 
the faculty. In hindsight, more celebrations of short-term wins could have taken place. 
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